Attention Allocates Vwm

Though it is clear that it is impossible to store an unlimited amount of information in visual working memory (VWM), the limiting mechanisms remain elusive. While several models of VWM limitations exist, these typically characterize changes in performance as a function of the number of to-be-remembered items. Here, we examine whether changes in spatial attention could better account for VWM performance, independent of load. Across two experiments, performance was better predicted by the prioritization of memory items (i.e., attention) than by the number of items to be remembered (i.e., memory load). This relationship followed a power law, and held regardless of whether performance was assessed based on overall precision or any of three measures in a mixture model. Moreover, at large set sizes, even minimally attended items could receive a small proportion of resources, without any evidence for a discrete-capacity on the number of items that could be maintained in VWM. Finally, the observed data were best fit by a variable-precision model in which response error was related to the proportion of resources allocated to each item, consistent with a model of VWM in which performance is determined by the continuous allocation of attentional resources during encoding. CONTINUOUS VWM MEDIATED BY ATTENTION 3 Public Significance Statement Visual working memory supports the maintenance of visual information “online” for short periods of time, and is known to be able to store a limited amount of information. The present study examined whether the distribution of spatial attention across items in a visual scene could account for changes in visual working memory performance independent of the number of items that had to be maintained. Several analyses from two experiments revealed that performance was best explained by the proportion of attentional resources allocated to each item, rather than by the number of items that had to be remembered. These findings inform our understanding of how the brain maintains visual information on-line to support everyday behaviors, suggesting that the amount of attention allocated to an item is a limiting factor in how accurately that item can be stored in, and recalled from, visual working memory. CONTINUOUS VWM MEDIATED BY ATTENTION 4 The ability to hold visual information in mind for short periods of time is critical to performing numerous everyday behaviors (e.g., remembering the number and location of cars in your mirror while driving), and has been linked to important aspects of cognition, including fluid intelligence (Unsworth, Fukuda, Awh, & Vogel, 2014). Despite the utility of this cognitive ability, one of the definitive attributes of visual working memory (VWM) is its severely limited capacity; only a small number of high-fidelity (i.e., high-resolution) representations can be maintained in VWM (Cowan, 2001; Luck & Vogel, 2013; Ma, Husain, & Bays, 2014); as the demands for VWM storage increase, there is an associated decrease in the number and/or fidelity of representations maintained. The exact mechanism of information loss in VWM remains elusive. The two predominant theories in this debate are the discrete-capacity (Luck & Vogel, 2013; Zhang & Luck, 2008) and continuous-resource (Bays & Husain, 2008; Ma et al., 2014) models of VWM capacity, although variants on these theories exist (Fougnie, Suchow, & Alvarez, 2012; van den Berg, Awh, & Ma, 2014). According to the discrete-capacity model, VWM can store a fixed number of visual objects, and once this number is exceeded all items not encoded in these few storage “slots” are forgotten (Luck & Vogel, 1997; Zhang & Luck, 2008). Further, changes in memory load can affect fidelity, but only when the total number of remembered items is below capacity, possibly reflecting the sharing of resources among slots when memory loads are low (Machizawa, Goh, & Driver, 2012; Zhang & Luck, 2008). Changes in load above capacity should generally not affect memory fidelity. CONTINUOUS VWM MEDIATED BY ATTENTION 5 By contrast, continuous-resource models posit that there is no upper limit on the number of items that can be maintained in VWM; rather, capacity is constrained by a limited pool of resources that must be allocated across all maintained items; increasing the number of items entails that each will receive a smaller proportion of the available resources, resulting in a proportional loss in representational fidelity (Bays, Catalao, & Husain, 2009; Bays & Husain, 2008; Wilken & Ma, 2004). As evidence of this continuous allocation of a shared resource, Bays and Husain (2008) demonstrated that VWM error (inverse of fidelity) varies with memory load according to a simple power law, consistent with the predicted change in noise of a neural population code (Bays, 2014). Most studies examining models of VWM place significant emphasis on the relationship between memory load, as defined by the number (and occasionally complexity) of the physical stimuli to be remembered, and memory performance. While this approach has been very informative, it overlooks the potential flexibility of this memory resource. For example, many studies have demonstrated that VWM performance is sensitive to eye-movements (Bays & Husain, 2008), presentation order (Gorgoraptis, Catalao, Bays, & Husain, 2011; Zokaei, Gorgoraptis, Bahrami, Bays, & Husain, 2011), attentional cues (Zhang & Luck, 2008; Zokaei et al., 2011), attentional lapses (Fougnie et al., 2012), reward (Klyszejko, Rahmati, & Curtis, 2014), and even voluntarily shifts in performance (Machizawa et al., 2012). Thus, VWM performance cannot be fully explained by item load alone, but must also account for situations where resources are distributed unevenly across remembered items. However, most studies have tended to assume that memory CONTINUOUS VWM MEDIATED BY ATTENTION 6 resources are distributed evenly, potentially failing to capture the variability among items (Fougnie et al., 2012; van den Berg, Shin, Chou, George, & Ma, 2012). One potential source of variation in VWM is attention. It has long been known that when trying to remember an array of letters, cueing attention to a subset of items in the array improves performance for those items (Sperling, 1960). Thus, restricting the scope of spatial attention to fewer items reduces the demand on resources and improves performance. Notably, the loss of encoding accuracy that occurs when attention must be distributed across multiple items occurs even in the absence of spatial attentional cues (Duncan, 1980). Consequently, when multiple items are to be remembered, it is potentially the distribution of attention that limits performance, regardless of whether explicit attentional cues are present or not; the number of available visual items to be remembered is simply a confounding variable over which participants must allocate attentional resources. Consistent with this account, several recent variable-precision models incorporate the allocation of attention during encoding into models of VWM capacity (Fougnie et al., 2012; van den Berg et al., 2012), suggesting that the allocation of attention, rather than storage limitations, may play a critical role in limiting VWM performance. While the attentional mediation of VWM resources is a prediction of most continuous-resource models, there are also two variants of discrete-capacity models that predict comparable roles for attention, with some key differences (Machizawa et al., 2012; Zhang & Luck, 2008, 2011). First, according to the slots+averaging model, the precision associated with each memory slot is fixed, however memory CONTINUOUS VWM MEDIATED BY ATTENTION 7 precision can be improved by storing individual items across multiple slots. One prediction that separates this model from continuous-resource models is that the least amount of resources that can be allocated to a single item is determined by the amount of resources present in a single storage slot. Second, according to the slots+resources model, there are no restrictions on how resources are allocated across memory items; the main difference that separates this model from continuous-resource models is that resources can only be allocated across a fixed number of objects. Accordingly, when the number of items to be remembered exceeds this item limit, the slots+resources model predicts that extra-capacity items are forgotten, whereas continuous-resource models predict all items are remembered, but the precision of each item is proportional to the amount of resources allocated to it. Thus, continuous-resource models and these two hybrid discrete-capacity models differ primarily in how they predict load and attention can affect the precision and likelihood of items being stored in memory. Specifically, while fixed-capacity models posit that the effect of attention on performance should be constrained by the memory load, as well as by whether or not items can receive less than one “slot” worth of resources, continuous-resource models (in particular variable-precision models) impose no such constraints on the effect of attention. However, while numerous studies have examined the effect of load on VWM performance, fewer studies have examined the effects of attentional allocation. In the present study, we examined how systematically varying the distribution of attention across a fixed set of stimuli influences the way those items are encoded in VWM. Participants saw memory arrays containing six (Experiment CONTINUOUS VWM MEDIATED BY ATTENTION 8 1a), or four or one (Experiment 1b) colored squares and, after a brief delay, were required to report the color of one item using a continuous response (Wilken & Ma, 2004). To evaluate the role of attention, we presented predictive spatial cues during the memory array, and systematically varied both the number and predictive validity of the cues. We specifically tested the prediction that if VWM resources are flexibly allocated via attention, then the proportion of resources allocated to each item should be best described by a power law that follows the predictive value of each cue, independent of the load (i.e., the number of items with a greater than zero percent chance of being probed). We also examined the specific predictions of the continuous-resource and discrete-capacity models, to determine whether this effect was limited to a fixed number of items. Finally, we compared the fit of different memory models to our observed data, to determine how well the results could be explained by fixed-capacity as opposed to continuous-resource models.